AIbase
Home
AI Tools
AI Models
MCP
AI NEWS
EN
Model Selection
Tags
BBPE tokenization

# BBPE tokenization

T5 V1 1 Base Ko
Apache-2.0
T5 1.1 model trained on Korean corpus, utilizing BBPE technology and MeCab morpheme analysis for optimized tokenization
Large Language Model Korean
T
team-lucid
18
3
Pko T5 Base
pko-t5 is a T5 model specifically optimized for Korean, trained exclusively on Korean data using BBPE tokenization to address Korean segmentation issues.
Large Language Model Transformers Korean
P
paust
874
19
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
English简体中文繁體中文にほんご
© 2025AIbase